Skip to content

OpenAI: preserve Chat Completions response_format when forwarding to Responses#1452

Open
zhuhaow wants to merge 4 commits intoWei-Shaw:mainfrom
zhuhaow:fix/chat-structured-output-forwarding
Open

OpenAI: preserve Chat Completions response_format when forwarding to Responses#1452
zhuhaow wants to merge 4 commits intoWei-Shaw:mainfrom
zhuhaow:fix/chat-structured-output-forwarding

Conversation

@zhuhaow
Copy link
Copy Markdown

@zhuhaow zhuhaow commented Apr 4, 2026

What changed

  • preserve Chat Completions response_format when requests are forwarded through the Responses API
  • keep structured output schemas intact for both OpenAI API key and OpenAI OAuth accounts
  • add regression tests for request conversion and forwarding

Why

  • Chat Completions requests were dropping response_format during Chat -> Responses translation
  • that caused structured output schemas to be lost before the upstream request was sent

Testing

  • GOCACHE=/tmp/sub2api-go-build GOMODCACHE=/tmp/sub2api-go-mod go test ./internal/pkg/apicompat ./internal/service -run 'TestChatCompletionsToResponses_ResponseFormat|TestOpenAIGatewayService_ForwardAsChatCompletions_(APIKey|OAuth)PreservesStructuredOutput'

@zhuhaow zhuhaow changed the title Fix chat completions structured output forwarding OpenAI: preserve Chat Completions response_format when forwarding to Responses Apr 4, 2026
@zhuhaow zhuhaow force-pushed the fix/chat-structured-output-forwarding branch from bc9dfc8 to 5b86d08 Compare April 4, 2026 00:49
@zhuhaow zhuhaow marked this pull request as draft April 5, 2026 09:06
@zhuhaow zhuhaow marked this pull request as ready for review April 5, 2026 09:55
@ddnio
Copy link
Copy Markdown

ddnio commented Apr 14, 2026

Nice fix for the OpenAI path — we hit the same bug in production (langchain-openai 1.1.x with_structured_output returned plain text and pydantic json_invalid).

One additional gap worth flagging: the Anthropic-platform path has the symmetric problem. After CC→Responses conversion, ResponsesToAnthropicRequest in backend/internal/pkg/apicompat/responses_to_anthropic_request.go doesn't consume req.Text.Format, and AnthropicOutputConfig in types.go only has Effort (no Format field). So even if this PR lands, a response_format through a group whose platform is Anthropic (handler: GatewayService.ForwardAsChatCompletions → Anthropic) still gets dropped before reaching Claude.

Anthropic shipped structured outputs GA on 2025-11 (output_config.format, supports type: "json_schema" and type: "json" — note json not json_object). Mapping is straightforward:

Responses text.format {type: json_schema, schema, name, strict}
  → Anthropic output_config.format {type: json_schema, schema}  // name/strict dropped
Responses text.format {type: json_object}
  → Anthropic output_config.format {type: json}

Docs: https://platform.claude.com/docs/en/build-with-claude/structured-outputs

Happy to send a follow-up PR for the Anthropic path once this lands, or include it here if you'd prefer one unified PR — let me know which is easier to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants